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- PREFACE 

^ Thi'S report covers the continuation of activity to improve the 
occupational competency assessment program in Pennsylvania. M*ny diverse 
requirements^were addressed along with new and sometimes different activity. 
When all activity is considered it represents a program that provides an 
important service function * i.e., the first step in ae^}uirfng voeatk>nal 
teachers in Pennsylvania. . . ' 

As with preceding years, the quality of the test instruments was 
reviewed and new testing developed. Changes and updates were made to man^ 
tests, through input of examiners and candidates, with considerable data 
developed at the examiner workshop. ^ 

The NOCTI national seminar was held at Penn State, in conjunction with 
the examiner's workshop. Great Interest In our program was expressed by 
participants from. the 22 states represented. Their exposure and Involvement 
with us, along with copies of our guidelines and forms Iprov^ldfed to partici- 
pants should materially assist in improving occupational competency 
assessment in other states. Many participants hatf^'considerable discussion 
with our test center coordinators as well as Individual examiners, tn which 
they attempted to identify solutions to problems in their states. 

Our involvement wit)i NOCTI continued at an even higher level than in 
previous years. Our visit to NOCTI in iJuly 1978 permitted greater under- 
standing of their operating conditions. With. NOCTI acquiring 9 of our tests 
fpr national use, we explored and consummated transfer of all of ouri ' 
Pennsylvania test bank (28 titles) to^NOCTI. • 




We encountered great cliff icultj' as a. result of N8CTI qhanging 
performance test scoring but using the' established data base. This 
generated extensive statistical studies for us until we found a solution. 



This, however, will be oft temporary nature ^until a new data base 15 



■I- 



established. • '"^ ' 

Cost studies for individual test centers resulted in funding' support 
by the Bureau of Vocational Education. This, wf 11 be cpntlnuing, aotiyltj^ 
to attempt- to Identify changes or adjustments tfiat wj 11 reduce i^he' oyera^l 
costs of the program. \ • '.' 
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REPORT OF PROJECT NUMBER 94-9013 

P. D. E. Occupational Competency Examinations 
July 1, 1978 - June 30, 1979 

INTRODUCTION 

Successful com|:^et'ion of occupational competency assessment is mandated 

by the Pennsylvania Board of Education as a requirement for vocational 

teacher certification in the trade and industrial program and for certain 

- . ■ - -v.. _ 

occupational areas of other vocational programs . This is viewed as necessary 

to insure that those persons who enter the classroom as a teacher are in fact 

competent in th€^ occupation they^wish to teach^ The program of competency " 

•assessment would. appear to be successful, in that a consistent pattern of 

successful candidates has emerged. To be admitted to the program, one must 

have adequate work experience beyond the learning period to establish 

competency in the occupation to be taught.' ^ven with these credential . 

requirements, we find a failure rate of approximately 25%, indicating that 

the test is screening those persons of poorer occupational preparation and 

insuring t(J a greater degree, the competency level of those persons entering 

i < 

the classroom. • . 

This project was designed to provide the management .a-ctivi ty of the 
overall occupational competency assessment program in support of the 
vocational education activity in the state of Pennsylvania and meets teacher 
edi|£at1on priority one. This includeis activity such as use of community 
resources' and recruitment of .vocational -teachers. .• Teachfer education 
priorities ,two and three, are also supported by this project. 

The project was funded under project Number 94-9013, from July 1, 197? . 
through June 30, 1979. Objectives were: ' * 



1. To serve as the Pennsylvania Coordinator of Occupational 
Competehcy Assessment and^o coordinate activities of the 
occupational competency assessment- program with* Temple 

Un versity. University of Pittsburgh, The PennsyJvania State 
University, Bureau of Vocational Education and National 
Occupational Competency Testing Institute aruUuch other 
universi,tie$ as may be assigned/ 

2. To maintain a master file of Pennsylvania developed and 
administered examinatijft and to reproduce and distribute' 
exaifli^^tions to the institu'tions and to the Bureau 
of Vocational Educa^^V^ 

3. To do a continuous item analysis of the results of all 
Pennsylvania developed and administered occupational 
competency examinations and to publish normative values 
for examination scoring. 

4. To provide in-service training for the test developers for 
institutions developing tests. 

5. To provide in-service training for Pennsylvania examiners 
of occupational competency examinations. 

6: Ta identify occupational competency examinations requiring 
^ revision or development through review of statistical 
studies. 



7. To>validate Qccupatfonal competency examinations develobed 
in the 1978-79 school year. 

8. To conduct ^ontihuous study of the occupational competency*, 
. evaluation program to identify changing needs and to 

recommend areas for improvement. 

Coordinate OC^ Program Activity 

Total program ^ictivity was coordinated through meetings, personal visits, 
workshops, conferences, and frequent contact*with the test center coordinators 
at each institution. Periodic meetings wereiheld at different universities, 
with meeting location- being centralized at tWe bureau* of Vocational Education 
for the second half of the reporting period. Use of a central meeting point 
is considered of great&r convenience to all participants. - 



A new publication "AppVoved Vocational Course Descriptions fOr Pennsylvania 
Secondary Schools" was completed, printed and distributed to all schools*^ 

Ik 

Presentations on use of the manual -was provided to" Pennsylvania Depart^nent of * 
-Education staff responsible for school evaluations ^as well as the three Bureau 
of Vocational Education field staffs and to regional'itleetlng^ of vocational 
administrators. The publication includes pertinent program, occupational 
competency and certification data required in the Conduct of approved 
vocational programs and is used as official guidance fop the conduct of 
vocational programs. ■ r - * . . - 

Greater involvement with the National Occupatioti,«l Competency' Teiting'' 
Institute (NOCJI) developed during thi-s^ reporting period. One- of the earliest 
items was discovered with results of fall 1978 performance testing, A far* 
greater failure rate than that which we were accustomed Weis ex^^ienced by 
the test cen.ters. Review of- the situation revealed that the difficulty wa^. ^ 
caused by NOCTI changing their performance evaluation format and applying the 
statistical base of ^he earlier format. We initially applied a temporary 
corrective factor by using' the average performance score as the acceptable . 
v^lue. Ttiis was found to be less^ than .sat isf2(ctory; therefore a further ' 
adjusted cut-off value of one standard deviation below the mean was adopted. 
This value was still used a't the fclose/of this rep'ort, however there appears 
to, be long range difficulty with this approac+i which wi'll require further 
study ta arrive at a long ran^e -solution. ^ ■ - 

A second major activity with NOCTi waffthe transfer of the entire 
Pennsylvania tes't bank.tb NOCTI, to become national tests^. The transfer of 
the test bank was initiated by NOCTI lo permit them, to acquire ninetes^. 
This was expanded to include the total Pef^nsyl vania test bank, in which ^ 

• ■ - « * 



^NOCTI would provide test bank management to include printing- and dissemination 

* ' * ' i ' 

scoring, norm, accumulation, revision and update'. 1t is expected tr^at 
•cons1deK»abl^^ additional ^effort and Involvement; on our part wiji be necessary ' 
to complete all requirements-in converting the instrumenty^^ational test 
status. Actual, transfer of the fnUruments took place on June 15i" 1979. 
The master l^ist of all testing Is shown on Table I. ' .: 

Review of test center administrative structure and cost effectiveness 

•." . . - , J . _ . ...... , , .. _ ...... 

involved .considerable study and time. Details of activity from' the time a 
candidate makes inquiry until testing results are disseminated have been', 
thoroughly reviewed and adjust^. Recommendation was 'made to permit greater 
secretarial involvement in routine activity and to free the faculty member 
serving as' the test center coordinator from all activity not requiring direct 
coordinator fRvolvement. Reduction in operating-costs sjioiild appear as a 
result of recommendations implemented. - ^ 

-—Assessments for the period of this report number 357. This is similar . 
to activity of the previous year and while giving the appearance of a down- 
ward trend when compared to two years prior activity, one should consider 
the .cyclic pattern of test activity. The overall average shows that we might 
expect 369 assessments per year for the entire state, however it appears that 

there may be a "larger than usual turnover of teachers for the 1979-80 school 

< ••• 

year. Averaging assessment figures would seem more appropriate when preparing 
plans and conducting studies. See Tables II, III and IV. 
NOCTI Na^tional Seminar : . 

The annual NOCTI seminar for state representaf^ives was hosted by 
Pennsylvania with activity 'conducted at The Pennsylvania State University. . 
Representatives from 15 states attended -the two day program with 34 persons- 
attending. The seminar was held in conjuncTtion with the Occupational 



Competency Examiner Workshop to permit parti-cipantS to gain knowledge of 
this part of our OCE activity. ' Information gairred may be used in reference 
to conducting similar activity in other states. A highlight, of the seiflinar . 
was a banquet presentation during which Dr. William A.' Williams, Professor 
Emeritus of Industrial Education, The Pennsylvania State University, covered 
vocational education and the development and evolution of occup^itional 
competency testing in vocational* education. 
Summer Test Cycle 

Summer* testing w.as scheduled at each test center to permit test completi 
for ft^rsons expecting to become teachers in fall 1978. The test schedule was 

t * V 

different for each test center, b.ut with a short time period in which to 

complete *tesfing, there did not appear to be a great difference in the 

schedule. All persons who applied to the test centers were tested at the. 

scheduled fee of $150.00.. In some cases, school administra^rs did jiot 

become aware' of resignations until after summer testing. These people were 

scheduled for fall 1979 testing. Review of results shows that summer testing 

permits many new teachers to enter the classroom on a vocational -intern 

certificate, rather than an emergency certificate. This. certification ^ 

status for new teachers is desired at all levels of vocational education and 

summer testing 'is key to this accomplishment. It is recommended that summer 

_j> ♦ 

testing continue to be offered for this purpose^ 
Test Reproduction and Dissemination 

All test materials used by tes,l^ centers were printed at The Pennsylvania 
State University and distributed to the test center^s as they were requested. 
Distribution -was by United Parcel SeWices generality', to hold co.sts down, 
'with special requests being shippecj/ through posta.l facilities. It was 



found that test center requests could j^e accommpd^jted with little difficulty; 
even though special requests sometimes required 'breaking the printer' 
schedule to print our miUterials ort' shor*t' notice .and special handl ing and ^ • 
hurried trips^to the main postal facility in town to meet a postal' dispatch 
schedule. This latter requirement is a necessity to support t^st center 
needs, particularly when testing of prospective teacher is not Iftade knoJn to 
a test center in the- normal schedule of activity. The test centers, in 
attempting to serve vocational education teacher placement, sometimes find 
it necessary to request special seryice, of this kind. 
Norm Data . ■ \ 

Written test optical scan sheets for each person completing a Pennsylvania 
test were submitted by each test center. These were generally copies of the 
original scan sheets, which necessitated copying to a clean sheet that could - 
be processed at the examination services Center, All data accumulated was 
processed through item analysis to obtain cjjrrent NORMs for test center use. 
It is essential that this data is current, since a determination of pass/fail ' 
for a candidate Is based on the mean valpe computed from the population^ « 
tested. Current NORM values are shown on Table V. 
Validation Studies 

Only one full test had been developed which required processing through 
validation studies. Review of the test bank revealed that two additional 
tests that haj^ been developed previously had not been fully val ida^ted." These 
are titles in which there are less than three programs in the state ^ it 
was for this reason that they had not , previous 1]/ been completed. ' Difficulty 
was, encountered in locating persons to complete the test, and limited data 
was obtained because' of so few programs being conducted in the state. Effort 

1 r 
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to overcome the lack of data was bejieved necessaV^ however, and since only 

one full test. was included -in ^l^ilf study; apiple .time was avai-lable to address 

the deficiencies of the other two titles. Statistical data^from the 

— --^ • • - • • 

y.al1dation studies will be published separately. . . 

Test Developers . • ^ V . . ^ • 

Persons, developing examinations at Penn State were provided workshop 
activity in test development processes. This "covered the full range of ' 
test development requirements outlined in the publication "OCE Examination 
Development Guidelirttes" . The initial meeting at which this activity was 
conducted was held at Harrisburg-Steel ton-Highspire AVTS in Harrlsburg, PA 
on Jan. 27, 1979. An additional meeting was held at Penn State to review 
these requirements with the persons not present at the Harrisburg meeting. 
Test developers at the other two universities were provided review of 
• f*equirements by the test coordinators tit the individual institutions. This '~ 
was considered npcessary and appropriate as attempts to schedule the 
activity in which the"persons would be centrally ^^ssembl^d were not successful .. 
Results of "tfiTs; activity will be realized through the input of tests- 
developed and the amount of additional work that may be required. The ; 
entire process will be evaluated to det^ine the best approach to prepare 
test develop^ers. 
Examiner' s Workshop • . 

The value of conducting an annual workshop in examining practices and 
. performance evaluation was realized several years ago. Our original goal 
was, and still is, to have each of our 300 plus examiners- from throughout 
the state attend a w()rkshop in which the entire examination could be 
reviewed by each examiner; where several examiner's in like occupational 

If) ^ 
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^Ititles^cOuld dl€cuss examining practices as they individually understand it;' 



y * 



and .to arrive at a standardized approach- to the .process! While we oViginally 
conducted a workshop with a lar^e attendance, we subsequently reduced the 
attendance to 60 examiners, w>iich has proven to be an appropriate number for 
a workshop of this kind. • \ . 

• ^ ^ The workshop this year was designed for 60 examiners, with eaclj^of the 
three test centers authorized space for two examirlers in ten titles. ^ • 
Experience has shown that all spaces are not filled by the"' test centers, so * 
arrangements were made to have the test centers also^allot spaces for all - 

0^ examiner's not previously attencjing, regardless of occupation. 

Pre-registration was considerably reduced, of which one reason given 
was that some schools had delayed closing due to making up lost days for 
sn6w and strides. A greater number of persons than could be accommodated 
were contacted and 56 responded affirmatively. Another difficulty was 
encountered in that fuel shbrtag^ was prevalent at the. time the workshop 
was scheduled and a number of persons who had pre-registered bailed to 
show because of the fuel shortage. This will have to be considered for 
future workshops and arrangements attempted to car pqol or to use public 
transf)ortation. 

occupational titles used in the workshop represented both NOCTI and 

Pennsylvania developed testing. All of the Pennsylvania testing had been 

transferred to NOCTI prior to the workshop, but not all- o# the tests had 

been converted to the NOCTI format. The Pennsylvania pepormance evaluation 

IT . ■ 

■ had. been converted to. the current NOCJI style for all excepftwo titles, 

however. Because of the recent transfer of testing, some tests used were 

» ■ ■■ _ — ■ ' 

obtained from Pennsylvania files wi'th a large number supplied by NOCTI. 
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the National Occupational Competency Testing Institute (NOCTI) as well 
as the Pennsylvania test centers participated by presenting pertinent topics 
and y^orking with the occupational groups. State representatives attendintk ' 
the National Occupational Competency Seminar, hosted by Penn State, \ 
parti JiVated in the morning session of the workshop.' They were qiv^n an 

- . » • \ %^ 

opportunity to discuss occupational competency practices with examiners and 
to become acquainted with the conduct of the workshop and the goals e*xpressed. 

Each of the written examinations were reviewed by examiners, in which 
they were asked to provide feedback on their understanding gf tf^condltion 
of the test. Comments will be used to improve the tests where appropriate. 

The larger time was used to review- the performance test. As in previous 
workshops, examiner's had the opportunity to discuss al.l pa.rts of the test 
and to obtain views 'that others may 'Have on each item. Standardization 
should be enhanced through the interchange of ideas and test improvement 
should result through the suggetstions ^^subjnitted. All comments were provided 

r ■ 

to NOCTI' to permit th,e^ to make changes to tests 'as appropriate. 

Considerable effort was made* tb rel/iew evaluation practices and the 
evaluation summary sheet introduced by l?OCTI within the past year. 
Advantages of the new form and the meffrpd of evaluating competencies were 
favorably received and should permit greater objectivity in arriving at test 
results. 

Individuals who participated and p/esented were 



Dr. Frederick Welch ^ . ' The Pennsylvania State 

University 

Mr. John Brandt ^ Bureau of Vocational 

^ * . ^ " . Education 

Or. Ray Klein • ...v ' ' NOCTI 



IS 



pr. Jon. Hunt , " ' Temple University 

* Or. Thomas O'Brian , * ' Indian^ University of 

) ' Pennsylvania 

i 

.. ^ Mr. Jor^v Lipps * University of Pittsburgh 

Mr.^ Gerald W. Funk - The Pennsylvania State 

' University 

Full details of preparation for and the conduct of the workshop are 

shown on 'Attachment' I. . 

Bureau of Certification Coord'i nat ion 

. , 

Coordination of OCE activity with the Bureau of Teacher Certification 
continued to W an effective process in achieving the goals established. 
Many problems, highly individualistic, were discussed throughout the year 
with resolutions usually a.chieved in short time ^Having a Bureau of Teacher 
Certification representative isit on the OCE consortium has eased the number 

* . -if 

of problems land has contributed materially to preventing situations exceeding 
their limited parameters. Coordination of activity at the working level is 
recommended for continuation. . 
Examination Development 

The test bank is, and has been,. under constant scrutiny. Tho^' titles 
showing greater use and greater te«>molog1cal change were identified and 
prioritized for test development consideration. Funds available for this 
development, under separate projects, were somewhat less than originally 
expected and resulted in a reduced schedule. Consideration was also given to 
changing priori ties, when 'it was (jetermined that applicants "rarely failed the 
oral assessment. 
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It was. agreed that test development funds should be used to prepare 
performance examj nation-is in occupations showing gflater use for which 
tbet-e were^no examination material s.*'*Each of the^-institutions- developing 
examinations wfer^ also assigned five pre-test instruments ernd five ' 
candidate experience checklists for'development. " " 

Test development assignments were: 
* University of Pittsburgh 

Performance Test, Pre-Test and Canplidate Experienc e Checkl ist 

'■ ' 5 ■ ■ 

• 17.0900 Commercial Photography ^ 
17.2391 Mine Equipment Maintenance 

17.1003 He?»vy Equipment - Construction - - 

Pre-Tests and Candidate" Experience Checklist 

» 17.0700 Commercial Art . ' 

17.12^00 Diesel 

17.1401 Industrial Electricity 

* 17.10t)7. Plumbing 

• 17-.2306 Welding 

\ ■ ■.. . - 

t Temple University , 

Performance Test, Pre-Test and Candidate Experience Checklist ' 

17.3301 Dressmaking 

17.1100 Custodial Services - 

17.9904' Vending Machine Repair 

Rre-Tests and Candidate Experience Checklist 

17.9905 Material Handling-' ■ " .* • 
17.1005. Painting and Decorating. 
17.3390 PoweA Sewing 

17.1503 " Radicr/TV Repair 

17.3100 Small Engine Repair .» " 

♦ 

* The Pennsylvania State University 

Performance Test, Pre-Test and Candidate Experience Checklist 

17.2802 Uw Enforcement 

17.0701 Interior Decorating 

17.0303 Auto Specialist (Service Station) 

Pre-Tests and Candidate Experience Checklists 

17.1400 Electrical Occupations • 

17.1501 Electronic Comm'uni cat ions 

17.0102 Heating 

17.1004 Brick Masonry " 
17.1093 Masonry Qccupat.ions 
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.... . ^ • • 

Continuous Study' of QCE Progf ^m 

■ * * ■ , ' ,. 

■ The, Pennsylvania Occupat1o^a^ Competency Assessment Program, 16 under 
constant scrutiny to identify trends "and changing patterns; to deve.lop 
.improved techniques anf* instruments^, to detennine and -Implement management 
improvements. Greater emphasis was placed/on secretary involvement at 
individual test centers to permit release time of the faculty coordinator 
for other activity- Test center .requirements and personnel responsibility 
are shown on Attachments II and III. * j 

Cost studies revealed that NOCTI could provide test bank custody and 
full services at only, a slightly higher cost than for us to manage it. 
Additional factors such as national statistics and national visibility were 
additional considerations in moving our entire tesj^ bank, to NOCTI. Another 
advantage in making this move is to reduce time and workload at the state 
level and direct resources- into other productivity. 

Working arrangements with-the state of Michigan will provide a sharing 
of test development and review w^ere1n we may benefit from resources that 
Michigan applies to test review and development. This sharing of responsi- 
bility should^reduce our t.ime cind funds allocation to maintain a valid and 
reliable test bank. 

' New performance evaluation summaries were developed for Pennsylvania' 
tdsts to match the change implemented by NOCTI. Acetate overlays to be 
used as a key for performance scoring were provided to the test centers. 

" These and other management improvement actions of recent years are 
shown belov/. 

Ma.nagement Improvement potions * . 

Standardize Test Development Practices 
Standardize Test Instruments 
Conduct Test Developer Workshops 
Conduct Examiner Workshops 

,12 



standardize te?t administration 
Establish state cut-off scores ^. . ■ 

Develop and update state policy manual 
•• - * Develop and update; Pennsylvania Departnient of Education 
' publication "Approved Vocational Course Descriptions 
for Pennsylvapla Schools" 1 ■ ' 
Develop acetaW overlay scoring kiys 
Develop pre-tests 

Develop candidate experientje checklists 
Re-fOrmat examination Instruments to national mode ' 
Develop candidate "Release of Information Statemen-t' 
Centralize test management * 
Establish consortium meetings 
■ V ^ Validate new test instruments , 
." Review test bank status annually 

Develop schedule to lipdate test instruments 
Develop master listing of examinations approved for 

Pennsylvania use 
Prepare consolidated listing of annual OCE assessments 
Prepare cpst studies , - • 

Prepare annual OCE projections 
Publish state approved statistical measures 

Summary 

The occupational competency evaluation "jj^ogram served the important 
aspect of providing the public schools with competent vocational teachers 
and" is a vital part of vocational teacher education, this testing 
program is the subjective and the most objective in all of the processes 
by which prospective teachers are evaluated. The activities described in 
this report express the interest and attention focused on these processes 
throughout the year. Development of new testing was continued by three 
universities;- Temple University, The Pennsylvania State University and 
University of Pittsburgh. Workshops for test developers were conducted to 
Insure 'standardization of the end product. Review of completed products 
and use of validation studies permitted improvement in individual test 
instruments. Statistical studies, using candidate testiiig results from 
the three universities has provided normative data that can be applied 

♦ ■ ' ■ . ' 

with greater reliability in establishing scoring parameters^ 
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'Ouality testing instruments are now available through central i.zed 

' ■ . • • ■ ■ ,« ■ 

^printing^ and -distributfon' by National Occupational Competency Testing Institute 

(NOCT'I).- Each university coordinator can provide ininediate diagnostic feedback 

to candidates who are. administered Pennsyl vani^a developed test^..' 

Of considiarable significance is the continuation of training ojf ' 
individual examiners, through a centralized workshop activity. This insures 
that standardization processes and .practices are known to all examiners ' 
attending and assists in the long-range goal of all examiners in the state 
receiving training. An important added benefit resulting from the examiX^r ' 
workshop provides a review of the individual tests for adequacy and technological 
changes. This had not been possible, except on a Timited baisis, pri6r to this 
workshop activity. 

Involvement with the National Occupational Testing Insti tute\{NOCTI) 
resulted in a continuation of test improvement ay^he ' national level. NOCti 
repres.entatives participated in the examiner worksT^op and provided detailed 
criteria for new performance evaluation which was introduced in 1978 on a 
national ^^ale. 

The-fentire Pennsylvania test bank was transferred .to NoK^in June 
1979 with |he agreement that they, NOGTI, would provide the same services and 
test bank management as with their original test bank^. ~ This |ction changes 
the test bank status .to a national level. Continued dialog has resulted in 
strengthening both programs. . , ^ * j 

Considerable effort was placed on publicizing the program throughout the 
state. . Presentations covering full details of the program were given to 

* 

Bureau of Vo'cational Education staff and to Vocational administrators at ^ 
separate meetings in the eastern, central and western regions of the state.. 
Presentations Vere also given at The Pennsylvania Vocational Education 
Conference. ^ 



We hosted the national seminar oyJOCfl member jstates, with'22 states 
represented. These , representatives participatedHn our examiner wofkshop 
And were provided with Pennsylvania OCE materials to assist in improving^ 
the OCE program in their states. The state of Michigan 'has been the most 
active of the NOCTI member states in establishing their OCE program. Ue 
have continued to assist the Michigan program which has permitted develop- 
•ment of a sharihg of test review and development. This sihould reduce 
some of our workload in this area. We will pursue this activity with 
other states to attempt to further share this requirement. 
' Considerable effort .andjfunding has contributed to a viable and 
quality program of occupational' competency evaluation for Pennsylvania. 
A continued effort fs needed to not only maintain, but to contintially 
Improve occupatijjrr'al competency evaluation," particularly to meet the 
needs of society and technological advances. , ' , 
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TABLE I 



MASTER LIST OF TESTING 



r 




L - l ehujh A - Aviilldble* 

__!>_. ^ IM-U « . Hat 
PS - PSU Avail able 

f - Templre / - Licen:»ure 
. X - Perfon«dnce 
Only 

lertlfiLtite ntle 

. A e r on (1 u t Tf. t» I I o c hn o I o« j y 
Air Condi t ton iiuj 
Aircraft Mtilntenance 
Aircraft Inijini? Mechanic 
Airframe 
AppI lance Repair 
Archltectural-Deslyn Tech. 



Audio-Visual XonnfiunltaHons Tech. 
Automotive Body & Fender ' 
Au t oiiio t i ve Mt'c ha n I c 



Automotive Specla'^llbt 



JO 



t 



AutoHiotlye re(^nolo(jy 
Auto-Wesel Mechanic 
Baker 

Building Construction Occ. 
Bulldinij Trades Maintenance 
Business Machine Ma int. /Repair 
; Carpentry 
Chemical Techno I o(|y 
Civil Technology 
^ Comniercfal Art ' ^ 

CoiniK}rcial 'Photography 

?g ' . 

*^icMmetology* 



o 

• O 

-3 O 



16.0101 
17.0100 
17.0401 
17.040102 



17.040101 

16.0103 
16.0691 
17.0301 
17.0302 
17.0303 
16.0104 
17.0390 
17.2901 
17.1090 
17. 1091 
-17.0600 " 
• 17.1001 
16.0105 • 
16.0106 
I7.O7IS 
17.0900 
.17.2602 



Use Ari^hl te(;tural Dtaftint) ^ 
Grtta ter Ratio 

FRir 



o 



3 



2501 
2001 
2503 
2604 
2502 
2003 
2105 
2510 
2005 
?007 
2028 
2008 

2009 
2014 
2013 
2015 
2017 
2101 
2103 
^^201 9 
2020 
2023 



to 



0 

A- 

Z 
Z. 

z 

A 

A 

A 

A- 

A 

X 

0 

0 

X 

A" 

A 

A 

a" 

0 
A 
A 

X 

z 



4J 
I/) 

4» 



C 

> 

a. 'x* 



p 



B 

B 



7/80 



A 
A 

A 

7/79 



7/80 



7/79 

r 

7/79 



CO IS 

Qt: X a. 



2/1 



2/1 



Multiple Numbers 
2/1 
2/1 



2/1 
2/1 



2/1 
2/1 
2/1 
2/1 
2/1 



VI 

2/1 
2/1 



t. 
o 



I 



.fi <^ 

O I 



Yes 



Yes 



Yes. 
Yes 



Yes 



Yes 



Yes 



Yes 



Yes 



cr 



Use NOCTI Air Cond. ^Refrigeration 



Use NOCTI Major Appliance Repair 



4- 



I - U'ht*|h A - Aval Idble 

P ^ IMtt - Not 
" PS - i^Sli Avii liable 

T - Icinph^ ^ * * it ensure 

X - Performance 

Certificate Title 

Custodial jervtLes 
Oicsel Moi.hanM 
Draftitu) Desitjn lechnoUnjy 
Oraftincj »Occuf)a t lon^ 
Drafting, Artln tec tura I 
l)rattin«|» Mechanical 
Drossniak i nij 

t lec tr icd I (Construe t ion/Ma int . 

Electrit;al, Industrial 

[lectrical Occupations 

L let trica \ fechnu lutjy 

i lec tronics 

i lectronic Techno I o<|y 

i I ec tronics CoiiHmjnlcat ions 

Uec tro-Mechan ica I I ech'noloqy 

[ M<J i neer i ni| F ec hno 1 0()y 

* tnvi ronnienta I Control Fech. 

Fal)ric Maintenance Services 

Kloor Covering) Installation 

Foufulry 

Graphic Arts 

Graptiit Occupat it)n'f 

Meatint) - * 

* 

Use Ura ftrni) Occupations 
Use I l»H tronics 



o 

• a> 

• O 



17. IU)0,. 

17.1 300 
I/. I.JOl 
17.1 JO? 
17.330) 
17. 1002 
)7. HO) 



17. 1400 
16.0107 
17. 1500 



16.0108 
17. 1501 
16.0109 
16.0100 
16.0110 
17. 1600 
17. 1094 
17.?301 
17. 1900 
17. 1907 
17.010? 



♦J 

ni 

o 

■r- <V 
«-> <— 



2520 
2025 
2027 
2027 
2032 
2034 
2029 
2031 
2036 
2033 
2107 
2030 
2111 
2110 
2109 
2104 
2113 
2037 
2074 
2039 
2057 
2076 
2040 



> 

o S 



♦J r— 

I -r- 



7/79 
7/79' 



P 
P 



A* 

A 
A 



P 
P 



7/79 
A 

7/79 



A** 

7/79 



7/80 



7/80 
7/79 



J I 

?5i 



(X. 



2/1 
2/1 

Multiple Numbers 
Multiple Numbers 
Multiple Numbers 



Multiple Numbers 



2/1 



2/1 
2/1 



2/1 



2/1 



2/1 



2/1 



2/1 



2/1 
2/1 



o 




Yes 



Yes 
Yes 
Yes 



Yes 



Yes 



Yes 



Yes 



Yes 
Yes 



Use NOCTI Printing 



Use NOCTI Printing 
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— p P i l l n , KiTi ' 

PS - l*SU • Avtil ldble 

f - leniple L - Licensure 

> Perfoniidnce 
On I y 

icrt H u<ite I Kle * 

Me<jvy hiuip- ions truotrln 
HydriUil lLs/Mii!(t I'owur 
IndustruH lechnolixjy 
Instruiiiont MtUntt-'Mdnci* A^ttepair 
ln'>tni«Hjifilii t ion ft* chnoloqy 
Industrhil Mii U^ter^ance Mech. 
Iriterior Denu d^uni 

Machine Shop Practice 
Machtrie Fool Oper*cJtion 

^ M*!sonr'y , IJr* icK 

D ... 

Mtisuriry Oci uptJ t ions 
Maloritil llafulMntj ^ ^ 
.Meat Cuttinij 

V 

MifchcWHca I l)r a t f. \ ruj/Dos i(jn lech. 
Mechlin » ccj I f echfU) I o<jy 
Me to I K<ihrtC()t Hin 
M<?tdHijr()i( a I FechnoUxjy 
Motdlwork(rt(| Occupations 
Millwork A rat)ir)e tnwik ino 
Mine t(|ul|Mnent Ma uUenance ^ 
Nuclear fochnolotj/ 

Use Mcthiinkol OrdfUrxj 

ERIC 



A 



o 

i 



17. U)()3 
W. 3i?90 
Ib.Oli) 
17.2100 
16.011i? 
17. 10<>2 
•17.0701 
17.200^ 
17.2302 
17.2303 
17. 1004 
17. 1093 
1 7 . 990r> 
17.2903 
16.0128 
16.0113 
17.2392 
16.0114 
17.2300 
17.3601 
17.2391 
16.0115 
17.2103 



8 



01 -t- 



2044 
2042 
2112 
2041 
2115 
2068 
2050 
2048 
2043 
2043 
2045 
2054 
2046 
2058 



2106 
2117 
2052 
2119 
2072 
2047 
26.30 
2120 
2116 



3 



X 
0 
0 
A 
0 
A 
A 
X 
A 
0 
A 
A 
A 
0 



A 
A 
A 
0 
A 
A 



X 
0 
0 



on 



O < 



•t- 



c 

r— 

c 

o S 

u. a. 



P 
P 



P 
P 



♦J r— 
CL t 



7/79 



7/80 



7/80 



7/79 
7/79 



7/79 



7/79 



7/79 



A* 

0 

'7/80 



7/80 



7/79 




2/1 



2/1 

2/1 
2/1 

2/1 



2/1 

2/1 
2/1 



2/1 



Multiple Numbers 
2/1 



2/1 



2/1 



2/1 
2/1 



t- 
o 



Yes 



Yes 



Yes 
Yes 



Yes 
Yes 



Yes 



Yes 



Yes 



Yes 



Use NOCTl Machine Trades 







X 














--- p Pitt" i) Not 

- PStI Aviifl<W>le 
F ■ k»iii|jlo / - I U en sort; 

X - PcM'formancf 
Only 

\ 

Cert If tcate Jitle 


c> 


c 

o 

♦-^ 

o 

•r- dl 
♦ -» r— 
L. 


3 
3 

in 


♦-» 
v\ 

» - 


ronr P 

Pennsylvania Test 


t/> .<a 
<u « 
f-» »~ 

1 -r- 
<l> «l> 
k. > 

CL < 


3 . 

pi 

u G V- 
a; D a> 

ot: i3E a. 


■ 

O 

1$ 

e a. 
R 3 

Ck! to 


Ptiiiitliuj tiful Uecoratini) 
> 


1/. lOOb 


;?049 


A 






7/79 


2/1 


Yes 


l\i t tornnuik \\\^} 


1 / 9903 




n 












Petroleum Product taii \ 


16.01 16 


' 2036 


0 








- ~ 





Pho t<>qV<iph 1 c fee hno 1 y 


16.0129 


2020 


0 








• 





fMastks 


W.^/OO 


2053 


0 


■ - - ■ 
— ■ — 





— 




- ' - - - 




— 


Plji>tK. fvchfu)l()t|y V 
V 


16.0^99 


' 2053 


0 












Plumbing 


17. 1007 


2055 


A 


0 




7/79 


2/1 


1 CO 


Power Mochdfiks Occupations 


4^ 1 7.0304 


2066 


0 












Power Sewint) 


17,3390 


2070 


A 


B 





7/79 


2/1 


Yes 


Quantity foods 


09.0203 


2059 


A 


- , 


n 

■ r 


7/80 


2/1 


■ CD 


i>oRddU)/lelovision Repair 


17. 1503 


2061 


A 




P 


7/79 


2/1 


— 

Yes 


Refrigeration 


1 / . 3000 


2024 


A 


B 

- 





7/80 


2/1 


Yes or 


ResLMrch I laboratory Assistant 


16.0690 


2080 


0 













School - Imp 1 oyinen t ' Coop, EcJ. 


17.9901 


2361 


0 






— . 


■ ^ 





Scientific l>t»ta Proc. (Coinp. I^ch. ) 


16,0117 


2121 


A 









2/1 




Sheet Mrtal * 


17,2305 


2063 


■A 


\ 




7/79 


2/1 


Yes 


Shoe Manuf <u: t'uf e/Repal r 


17,, 3401 


2065 


0 


■ - 










Smal 1 hi()ine Repair 


17.3100 


2067 


A 




n 

V 


7/79 


2/1 




lailorhu) 


17,3302 


2069 . 


X 




P 


7 /SO 


2/1 


Yes 


[ext i le l^ro(tu(. t iotv/l abr fw^HI^^I 


i 17, 3300 


2071 


A 


B 




7/79 


2/1 


Yes 


lool and Die M(»klfHj W[ 


- 17.2307 


J2073 


A - 




P 


7/80 


2/1 




Tool and Die Design Technology . 


16.0192 


2123 


A 













U|)h(5lsterin() 


1 7 3600 


2075 


V 

A 




P 


A 

M 




Yes 


Vending Mac hi we Repair 


) 7 , 9904 


2016 


X 


> ■ • ■ - 


P 


7/79 


2/1 




O limi 

Hm^aBle/lfidustrJjj,!, Other 1 


1 7 . 2306 
17.9999 


2079 


A 

. « . 

0 


q 

v^. — w)- 





7/79 


2/\ 
f ^ 
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Table II 



0. C. E. P. ADMINISTRATION FREQUENCY 
July U 1978 TO ; June 30. 1979 
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TOTALS FOR PENNSYLVANIA 



4. 



3 Page Grand Total 



357 



88 149 41 



14 



TITLES 



Air Conditioning/Refrigeration 



Aircraft Maintenance 



Airframe-Aircraft Mai ntenance 



Aircraft Engine Mechanics 



Appliance Repair 



Architectural Design Technology 
Architectural Drafting 



Audio->Vi$ual Communications Tech, 
Auto Body 



Auto-Diesel Mechan.ic 



Automotive Mechanic 



Automotive Specialist 



Baker 



Barbering 



Brick Masonry 



BuildiBg Construction Occ. 
Building Trades Maintenance 



Carpentry 



Commercial Art 



Commercial Photography 
Cook/Chef 



Computer Technology 



Cosmetolog y 



Custodial Services 



Sub-Totals - Page 1 



34 



CO 

a; 

r— 

^ O Ul 
< 



2! 

•r- 



51 



18 



34 



15 



16 



24 



15 



16 



9 



34 



16 



24 



15 



168 



-__2.i..._J 



45 



98 




16 



16 



3, 

c 

o 




FROM 



0. C. E. P. AOMimSTflATION FREQUENCY 
July U 1978 1- TO • June 3,0. 1979 



TOTALS FOR PENNSYLVANIA 



TITLES 



Dental Assistant 



Dental Laboratory Technician 



Diesel 



' Drafting Occupations 



D rafting,. Mechanical 



Electrical Construction/Maintenance 



Electrical, Industrial 



Electrical Occupations 



Electro-Mechanical Technology 

Electronics 

Electronic Comnuni cations 



Fabric M aintenance Services 
F oundry 

G raphic Arts 



Health Assistant/Med ical Assistant/NA 
Heating • 



iA 

-a 
c 

I— </> 
O (/> 



13 



8 



11 



12 



CO 
.3 



8 



8 



8 



u 



> 



O 



12 



Heavy Egiiip ment/Co nstruction 

Hort iculture 

Machine Shpp 



18 



18 



Masonry Oc cupations 
Materials HandT-ing 
M echanical Technology 



Medical Assistant 



Medical Lab Assistant 



Sub-Totals - Page 2 



35 



108 
25 



26 



38 



22 



FROM 



0. C. E. P. ADMINISTRATION FREQUENCY . 

-Jul y 1 1 . 1 978 TO June 30, \^7^ 



TOTALS FOR PENNSYLVANIA 



0 

ERIC 



: — 

/ 

• 

• 


ToUl Candidates 
Assessed 


Failures 


NOCTI 


Licensure 


Credential Review 


t 

A . ^ . ^ 

* • 

1 


Mi litary Science 


2 

1- C 






9 
c 


* 




Mil 1work/Cat?1netmakinq 


4 


1 
i 


A 
*t 






* * 
V 


Nurses Aide 


1 

1 1 






1 






Painting ^«nd Oecoratina 

» . — ■ ■ J - 


1 ■ ^ 












Plumbing . 




2 


A 








Power Sewing 


2 












Quantity Foods 


11 ' 


1 

1 








■> 


Radio/TV 

'■ [ — . 1 — • 


4 

1 ^ 












Refrigeration 


1 




* 








Restaurant Practice 


3 












Sheet Metal 


- 6 


1 


5 








Small Engine Repair 

— — J — - - J 


2 












Tailoring • 


3 








1 




Textile Prbdu^ion/Fabrication 


7 












Tool and Die Making 

; ■ ■■ ■ 1 


1 












Upholstering 


2 












Waiter/Waitress 


3 












Warehousing 


2 












Welding - • 


20 


12 










Mine Eguipment Miintenance 


1 ^ 








1 
































. * 1 














' r 

i« 1 








• 






, Sub- Totals - Page 3 30 


81 ^- 


- 17 


M3 




2". 


ir. . - .. • • 

f 











or 

it 




t 








< 


r 

Institution V 


Candidates 
Assessed 


* 

Tests 
Administered 




Fa i lures 


^ . 

Licensure 


Ora 1 

Examinatldn 


The Pennsylvania State University 


98 


80 




15 


3 


Temp 1 e Urn ve rs i ty 


159 


,119 


33 


22 


18 


University of Pittsburgh 


108 


81 


27 


24 


3 


Indiana University of 
PennsyTva^nia 


4* 







3 


1 


THTAJ Q 


369 


280 


74 


— ^ ^- 

64 


25 


\ 


\ 











OCCUPATIUNAL COMPETENCY 
FOR PERIOD JULY 



ASSESSMENT PROGRAM FOR PENNS 
1. 1976 THROUGH JUNE 30. 1979' 





* 

Iristi tuth)r) 




FotiU 
U)didtU 


es 




Total 
Tests 




< 

Faj^ures 




NOCTI 
Tests 


Licensure 


Oral 
Examination 






FY-77 


FY' 781 


rFY-79 


FY- 77 


FY-78 


FY-79 


FY-77 


FY-78 


FY-79 


FY-77 


FY-78 


FY-79 


FY-77'fY-78 
1 * 


FY-79 


■ — ' r- 

FY-77 


FY-78 


Fy-7^ 


» 


"The Peiwisy Ivtin icJ State 
Uinversi ty 


100 


110 


83 


84 


77 


78 


9 


17 


15 


30 


43 


38 


15 




4 


1 


6 


1 




reinple University jjj^ 


190 


130 


156 


141 


87 


128 


43 ' 


20 




78 


40 


60 


21 


24 


2Q 


28 


19 


8 


ro 


University of. ^jiP 
fM ttsbunjh 

Ifutiorid University of 
Pennsy ) vaqi-d- 






IH 

4 


80 


66 


96 


25 


19 


37 


30 

i 


28 


51 


41 


16 


14 

3 


4^ 


3 


4 
1 




TOTALS 




3*26 


357 


305 


230 


302 


77 


56 


88 

t 


138 


111 


149 


77 


67 


41 


33 


28 


14 



NOCTI - % of T ests /Tdniilnlstered 
flzJl - 78 FY - 79 

• ■ . 



FY - 77 



I^sts Adjnn|ster;Gd Kail 
FY - 78 




77 
'305 



25.2% 



56 

m 



24.3 , 



302 



29.1'X 



7 



Table V 
CURRENT NORM VALUES 




ERIC 



Ai J '.)tulf t \i\u Hui |/|8 

A()(>| Mtu o Krpj ] r * 10 

ArLliUoL turd 1 Drijftinq/ 

i(|n Km hnoJoyy 34 

Add in- V i i>ua I iDiniiiiin k a ( ions 

TiH firu) l()i)y 9 
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ATTACHMENT I 



OCE examiner; S WORK^SHOP 

The Penrftylvania State University 
University Park, Pennsylvania 

In Consortium With 
University of Pittsburgh, Temple University, 
and Indiana University of Pennsylvania 

June 21-22, 1979 ' 



A workshop for persons serving as occupational cQmg^tency evaluation 
examiner's was conducted at The Pennsylvania State University, University 
Park, PA on June 21-22, 1979. This workshop, one of many activities 
x:onducted in the continuing effort to improve the overall occupational 
ccdnpetency evaluation program, was patterned after previous examiner 
workshops with certai|] iDodifications . 

Whi'le the workshop was conducted at University Park, PA, it was the 
result of a coordinated activity itivolving University of Pittsburgh, Temple 
University, Indiana University of Pennsylvania, The Pennsylvania State 
University, and the Bureau of Vocational Education. Planning was continuous 
throughout the year, with details being finalized at quarterly policy 
meetings of the OCE coordinators from each of the four universities and the' 
Bureau of Vocational Education. ■ * ' 

It was cjecided quite early to limit the number of occupations to twenty 
and to schedule sixty examiners for attendance. This would permit us to 
schedule suitable meeting space and would insure maximum utilization of 
time. Experience with previous workshops of this nature show that we 
might expect better results by limiting participation. 
^ The agenda was drafted at the December 1978 OCE coordinators meeting 

and the twenty occupations tentatively identified. As work progressed in 
developing the workshop, it was realized that a considerable number of 
examiners had not previously attended a workshop. The goal was then 

erJc V . / ^9 ' . * 



changed, with the attendance geared to scheduling flis many examiners as 
possible who had not previously attended regardless of the number of 
occupations represented. 

PreVlpus experience had shown that we would not be able to attract all 
examiners who should attend, therefore we proceeded to irwite all persons 
identified. This numbered considerably greater than sixty, but as with 
previous workshops, we exhausted the list before reaching our attendance 
goal o^f 60.. which then nexressitated atWit tonal effort to attract the 
examiners. 

A- 

When the "'attendance list was finalized, we found that 22 occupations 
were to be represented. The agenda and the occupational .titles are shown on 
Appen^i/x 1 and 2 with examiners who attended shown on Appendix 3. 

The workshop wa§ schedulerd as late in June as possible, without 
conflicting with other scheduled activity, so that we might accommodate 
late jSchool closings due to make-up days for snow Conditions. It appears 
that. a date between June 15 and 25 will accommodate the larger number of 
peop'le. Our experience with this and previous years show that a firm date 
for activity of this type be assigned at an early time and that the date ; 
remairr firm. Changes to accommodate some school calendar adjustments only 
^ result in disrupting planning of individuals and has been counter- 
productive in efforts to improve attendance. For every person accommodated, 
it appears th^ a greater number are hampered. 

>lotif ication of the workshop. Appendix 4 was mailed to examiners, 
whose names were submitted by each institution. A follow-up notification 
^ was mailed to those not responding in' an effort to havens many attend as 
could meet the schedule. Based on responses, each university su^itted 
additional names, made many telephone -calls and exerted considerable effort 
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to fill their allotted spaces. Fifty-six persons were finally sclieduled to 
attend, t?ut fifteen of these failed to show. This was caused primarily by 
the gasoline shortage which occurred at the time the workshop was scheduled. 
Even by car-pooling^ we sitill experienced people who were unable to obtain 
gasoline 'or were reluctant to chance fuel problems. We will have to 
devise a strategy t6 overcome this, difficulty prior to the 1980 workshop* 
Workshop Facilities 

Facilities of the Penn State Conference Center wertf used for the workshop 
Meeting room accommodations proviai| large individual tables for each ~ 
occupational title. Full conference facilities such as a sound sVsten), media 
aids, air. conditioning, and generous space made the conduct of workshop 
activity considerably improved over previous workshops. 
Housing > . ' 

Housing for participants was made available in West 'Dorms, on campus, 
(same vicinity as the workshop). Each participant was authorized two nights 
lodging ^1n a .double r6om at $5.50 per night, to be paid through the wo^kshop 
funding. Participants could use single accommodations at $1.20 additional 
,charge„, to be paid by the participant. Arrangements were made for family 
members to accompany the workshop participants, with housing costs a 
responsibility of the participant. Several persons took ^advantage of this. 
Workshop participants were also given the option of securing off-campus 

accommodations at their expense and receiving a $5.50 per night reimburse- 

•. ' ■ ■ ' ■ ■ • . 

ment ($11.00 ma)/imum) upon presenting their motel receipt. A small number 

of persons used of f -campus \faci 11 ties. Use of the dormitories on campus 

was well received by participants and Is recommended for future workshops. 
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Meals ^ X 

. . \ 

Each participant was responsible for their^ own meal requirements. 
Campus eating facilities at the Kern Graduate Center cafeteria and the 
Hftze] Union Building Terrace Room were recommended. They were close tQ^ 
campus housing and offered ^ variety of nourishing food at nominal cost. 
Participants were reimbursed for meals via their expense voucher submitted' 
following the workshop. 

This method of meal ^econwoclatioi» permitted flexibility for part tc+- 

pants and reduced involved processes of meal tiqkets, schedules, etc. It 
was controlled by setting dollar limits and briefing participants on their 
responsibility. \. 
Parkjnil 

Campus parking is rigidly .controlled at Uoiv^ersity Park, the Conference 
Center parking area was assigned for parking for participants and was 
adequate. Difficulties encountered the previous year,' in which one 
conference activity monopolized parking td tJie dejtrlment of other conferees 
did not materialize. Corrective action by Uie Conference Center was 
effective; ' ' ' 
Registration 

Workshop participants registered for housing at Waring Hall' in the West 
DormUory area, as they arrived on Sunday afternoon and evening. A welcom-Wg 
letter, Appendix 5, along wlth^ocal and campus information was given to / 
participants as they arrived. j 

' Workshop registration was completed at the Conference Center registration 
desk^ under the direction c^f Mr. Robert Beam. • Three secretaries from the 
Division of Occfupational and Vocatiola^ Studies, were able to process all 
persons, in an orderly fashion and^without difficulties. 



NOCTI National Seminar ' 

The annual NOCT I National Seminar was hosted by Pennsylvania on 
June 20-21, 1979 with ^ssions conducted at The Penn3ylvania State 
University. The Semina^ was scheduled to permit state OCE representatives 
to also attend tJje Examiner's Workshop as a part of the Seminar sthedule. 
Participants and the states they represent are shown on Appendix 6," 

Seminar participants attended the first half day of the Examiner's 
Workshop and were given an opportunity to observe and discuss issues with 
individual examiners as well as to become better acquainted with the test 
instruments. The workshop agenda was developed to accommodate this 
activity and app.eared„ to meet the goals established. Seminar participants 
were provided a package of reference material relative to the Pennsylvania 
OCE program to serve as a' reference in their hotpe state program. Comments 
of participants and examiners were favorable for this activity. 
Program » 

The program developed at the December and March Coordinators meetings 
was folibwed with minor modifications, (Appendix 1). Both Pennsylvania and 
NOCTI te^ts were used"; NOCTI test materials for examiners where NOCTi tests, 
are being used and Pennsylvania tests for examiners in which these titles 
are being used. All Pennsylvania tests had been- transferred to NOCTI ;in 
May, 1979, and tTiis gave examiners an opportunity to become familiar witM - 
the changed format. 
Security ' 

Security df test materials required that each examiner sign for testing 
materials that'were to be used. These were not permitted to be removed from 
the workshop location and were inventoried when returned. The workshop. 

t ■ ■ ' . . ' 

.location was- under.constant surveillance or was locked when activity was 
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not scheduled. Provisions fo^r security were considered adequate, as all 
material was returned to files. 

This part of the workshop required two secretaries to sign-out material 
and account for it at the completion of the workshop. Use of two people 
was necessary to prevent delay in the start of activity. 

Examiners for each ^i tie represented were grouped at sepa»|)te tjables. 
Each group was asked to identify a key person or team leader, and to use his 
-f>^r -her ex«m+mrtfon^ materiai-s "js th^ master. Jet , to be f^tUTngd wtth comments 
from the group. Comments were made directly on testing materials to ease 
control of documents. 
Written Examination 

Examiners were asked to review the written examination in its entirety 
and to identify changes that should be considered. Considl^ation was to be 
giwen to the overall content by occupational categories and proportioning, 

* 

grammar, editing, currency, technological adequacy, and any other comment • 
that might improve the produqt. Since time was limited forthis activity, 
examiners were'^asked to assign sections of thQ test to different individuals 
so that the total test might be revieyved; 

Guidelines for test review that had been developed prior to the 
workshop were used for the first time and appeared to as^sist in the review 
process. Guidelines are shown on Appendix 7. ' • 
Performance Test 

The greater time was used to review and discuss the performance 

examination since this Is exclusively administered by the examiner. (Written 

tests are central ly^^dminl^s t'e red by each university). 

Mr. Gera,ld'W. Funk, Penn State, covered the workshop orientation and 

\ * 
test, development, and t)r. Ray Klein conducted the NOCTI part of the 



sessions. Presenters as shown on the agenda participated in the NOCTI 
seminar activity and were unable to remain for the, second day of the 
examiner's workshop. Because of this, these presentations were cohducted by 
Mr. Funk. Cons-iderable time was used in developing a rationale for evalua- 
•tion and those elements cons idiere<l_i"ipor tan t in the evaluation process. 
Dr. Ray Klein, NOCTI Industrial Psychologist, discussed statistical data 
and studies as well as development of evaluation criteria. Presenters 
worked with individual occupational groups throughout the workshop, to 
assist in greater understanding of the program and the role of the' examiner. 

By reviewing the examination, each examiner was expected to become 
more familiar with the product. They were given an opportunity to discuss 
exartiining practices with other examiners of the same occupation, as well 
as to ask questions and obtaib clarification on individual items or 
situations jof difficulty. Standardization was expected to be enhanced 
through tWs process along with the presentations on standardized examining 
pracfices. , 

The review process included evaluation of 1) applicability of each 
job, 2) the description of the candidate's instructions,, as well as similar 
instructions, considerably expanded for the examiner, 3) itemizing tools, 
equipment and materials needed for each job, 4) consideration of time 



allowed, and 9^ scoring factors and score for each in relation to the 
other scoring factors for the job and in relation to the total examination. 

Review of the examination was intended to permit each examiner to 
become familiar with parameters shown above and to develop a degree of 
standardization for^ach job in each examination. We would expect that 
exposure to conditions and requirements at different examining points, as 
well as skill and understanding of different examiners would provide that 

ERIC ^ ^^^c-^ \ . ^ 



leveling necessary for standardization. We would also expect an improvement 
in the parameters for each job which would result in overall test 
improvement. 

Varying ccxnments were submitted by the different groups, with some 
being morelcomplete than other^. Sufficient comment was received from each 
group to permit improvement in the overall p.rogram. 
Summary 

"Comment from examiners would indicate that the workshop was needed. 
Input from each of the groups will be used to update examinations, as 
necessary. Comment will become part of each examination mainterjance file, ' 
to be used by content persons improving testing instruments. 

We might expect improvement in examining practices, in the relationship 
of examiner to examinee, a better understanding of examination details aijjl 
in improved reporting. Consideration of these factors alon^ would indicate 
attainment of workshop goals and when "added to the data submitted on the , 
conditioning of individual examinations would indicate a successful 
workshop. ■ ' 
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EXAMINER'S WORKSHOP AGENDA 
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. ^ . APPENDIf 1 

OCCUPATIONAL COMPETENCY EXAMINER'S WORKSHOP 

* ♦ 

June 21 aftd 22, 1979 " ^ 



Thursday, June 2K 19 79 - 

8:00 - 9;00 Registration - Keller Conference Center - PSU 

9:00 - 9:10 Welcome - Dr. Frederick Welch 

9:10 - 9:30 Introductions - DCE Ortentation - Gerald W. Funk 

9:30 - 9^45 Recent Changes in Test Bank Status - Geralct-V^r' Funk 

9:45 - 10:00 &real( 

10:00 - 10:20 Overview to Pennsylvania Test Development - Gierald W. Funk 

10:20 - 10:40 Test Administration - Dr. "Jon Hunt 

10:40 - 11:00 Issues - John Lipjys ' ' . "» 

11:00 - 12:0(5 NOCTI Research and Development Studies - Dr^ Ray Klein 

12:00 - 1:00 Lunch . ' ^ 

1:00 - 1:30 Examiner - Candidate Relationships - Dr, Thomas O'Brian 

1:30 - 2:15' Review Written E)^aminations -' Gerald W. Funk. 
2:15 - 2:30 - Break , v . 



2:30 - 3:00 ^ Instructions for Administering Performance Examinations 

g Dr. Jon Hunt , 

3:00 - 4:00 Review Performance Examtnations - Dr. Thomas O'Brian 



Friday, June 22, 1979 



9:00 - 


10:00 


10:00 - 


10: 15 


10:15 - 


11:00 


11:00 - 


12:01/ 


12:00 - 


1:00 


1:00 - 


2:00 


2: 00 


2^15 


2:IS - 


3:30 


3:30 - 


4:00 



Performance Evaluation - Dr. Thomas O'Brian 



> Break ^ 
Statistics, and Evaluation - NOCTI - Dr. Ray Klein 
Review Performance Test - John Lipps ' . 

Lunch V 

Revlew Outstanding Sections of Test - Gerald W. Funk 
Break * 

Prepare Reports aid Recommendations - Dr. Jon Hunt 
,Wrap-Up, Turn in Materials -*Gerald W. Funk 

erJc ■ 5s ^ ■ '. " 
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APPENDIX 2 

eXAMINATION TITLES FOR EXAMINER'S WORKSHOP 
, ' June 21-11, 1979 ^ 



I. 


Air Conditioning/Refrigeration 


2. 


Architectural Design Technology 


3. 


Auto Body Repair ' 


4. 


Auto Mechanics 




> 

r 


• 9. 


Brick Masonry 


6. 


Building Construction Occupations 


7. 


Building Trades Maintenance 


8. 


Carpentry 


9. 


Diesel Mechanics 


10. 


Drafting Occupations 


11. 


Electrical Occupations 


12. 


Electronics 


13. 


Graphic Arts 


14. 


Law Enforcement 


15. 


Machine Shop 


16. 


Mi llwork/Cabinetmaking 


17. 


Plumbing 


18. 


Quantity Foods 


19. 


Sheet Metal 


20. 


Smal 1 Engine Repai r 



21. Textile Production/Fabrication 

22. Welding ,- - . ' 



■ 

46 



/ 



APPENDIX 4 

EXAMINER'S WORKSHOP LETTER OF NOTIFICATION 



ERIC 



61 



The Pennsylvania State University 

COLLEGE OF EDUCATION 
Dtvij^ion of Occupational and VociKtonal Studies 
RACKLEY BUILDING 
UNIVERSITY PARK. PENNSYLVANIA 1680: 



Each year, we conduct a two-day workshop oh the Penn State campus 
at University , Park or occupational competency examiners throughout the 
state. To date we have had, 218 examiners attend the workshop. As a 
part of OCE program improvement efforts our goal is to have 100% 
examiner participation at the workshop with attendance at least once 
each five year period or Vhen an examination is revised. 

Information available indicates that you are now, or will be, 
serving as an examiner for the university servicing your region of thfe 
state. We would like to meet our goal and invite y^r attendance .and 
participation in the workshop which is being held oPjune 21 and 22 
this year. Your travel expenses, lodging on campus^ and meals will be 
reimbursed and we will be able to pay ydu a $50 honorarium ($25/day) 
for the two days of the workshop. We will meet in the Keller Conference 
Center, which provides us with a modern, air conditioned and media 
equipped facility conducive to maximum participation. 

The OCE coordinators from Temple University, University of 
Pittsburgh, Indiana University of Pennsylvania, and The Pennsylvania 
State University will be presenting pertinent topics along witk Bureau 
of Vocational Education and Natignal Occupational Competency Testing 
Institute (NOCTI) participation and input. We also expect bureau of 
Certification participation as well as key OCE persons from other* states. 
There are many people requesting attendance each year, however, first 
priority goes to our examljiejrs. 

Would you please complete and return the attached form at your 
earliest convenience, indicating your plins to attend. We will mail 
full details at a later da^e to all who. plan to be at the workshop. 



Respectfully, 




Gerald W. Funk 
Pennsylvania Coordinator of . 

Occupational Competency Assessment 
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Please complete and mail this' form to: 



Gerald W. Funk 

Division of Occupational and Vocational Studies 
The Pennsylvania State University 
no Rackley Building 
University Park, PA 16802 



PLEASE PRINT 



Name 


(H^fflWv Address) 


Number and Street 


City Ccnmty 


State Zip 

■ T 


Social Security No.. 


Home Telephone 


w ■ ' ■ ' -^^ — 

Business Telephone 


Position 


Institution 


(Address) . 

1 


Number and Street 


City 


State 


Zip 


CIRCLE APPLICABLE WORn(S) 


• 





I ( do do not ) plan ta^attend the OCE Examiner Workshpp at The Pennsylvania 
State University, University Park, PA, June 21 and 22, 1979. 

I will require a (single) (doujble) toom for the evening/s of 
Wednesday, June 20 7 Thursday, June 21 ^ / Friday, June 22, 



I will arrive at approximately 



on 



(time) 



(date) 



I will check out at approximately 



on 



(time) 

I will make my own commercial accommodations: Yes _ 

r I 

r will require, accommodations for spouse and 



(date) 
No 



children . 
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OCCUPATIONAL COMPETENCY EXAMINATION WORKSHOP 

EXAMINER'S WORKSHOP 

J. Orvis Keller Conference Center 
The Pennsylvanija State University 

Jun^ 21 and 22, 1979 



WELCOME " to the 1979 OCE Examiner's Workshop! 

We should like to make your attendance enjoyable as well* as profitable. 
Professional development is considered a key element as well as improved 
standardization of examining practices a.cross thB state, fou will find 
examiners in your occupation from areas administered by Temple, Penn State and 
the University of Pittsburgh. Indiana University of Pennsylvania (lUP) is a 
, test center and will participate in this workshop, however University of 
Pittsburgh exartiiner's provide- service for lUP at this time. Through discussion 
and sharing of ideas, we believe that the total OCE Program will improve to the 
benefit of all concerned. 

We plan to rev tew- both" the written and the performance examinations, with 
the preponderance of fime used on the performance examination. The examinations 
will be reviewed in their entirety and you will have an opportunity- to selectively 
review and comment on specific areas. You will' have an opportunity to prepare 
draft material that may be used to improve examination effectiveness. Guidelines 
are included in your workshop packet and you will be provided directions in each 
•of the presentations. If you have questions, please make them known to a monitor 
from one of the spCnsoping institutions so that you may gain maximum production 
from your efforts. 

Coffee breaks are scheduled for both Th,ursday and Friday (see agenda), and 
will be in the lower level of the Conference Center. Lunch is scheduled for 
12:00 to 1:00 p.m. each day, with each person responsible for his/her require- 
ments. The Kern Graduate Center cafeteria H suggested for lunch, since it is 
across the parking lot from the Conference Center. 

We will be using examinations from the National Occupational Competency 
Testing TT>s4rjtute (NOCTI), as well as Pennsylvania developed examinations. 
Processes arm examining practices for both sets of materials will be presented. 
It is importynt that you be familiar with NOCTI requirements as all of the 
Pennsylvania examinations were transferred to NOCTI in May of this year. The 
same processes of evaluation apply to all testing, with the performance 
evaluations changed to the NOCTI format. 

Presentations and work sessions will focus on common problems in examining 
practices. Comments will be solicited and you will have an opportunity to ask 
questions. You v)ill also have the opportunity to share suggestions with the 
entire group in an effort to improve examining practices. 

We Should also like to provide time on'Friday afternoon to wrap-up work 
from the workshop and to prepare and submit your written comments and suggestions 
for f>rograro effectiveness. 
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Youir participation in this workshop is recognized by many persons as 
representing a sincere dedication to your (Profession. It is through efforts 
such as yours, that advances may be made for the benei4-fe of vocational education 

Thank you for participating.' - • ^ • . . 



Bureau of Vocational Education 
The Pennsylvania State University 

Temple University 
University of Pittsburgh 
Indiana University .of Pennsylvania 
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NATIONAL OCCUPATIONAL COMPETENCY TESTING INSTITUTE 
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GUIDELINES FOR REVIEW OF NOCTI E)(AM$ 

Technological developments, toupled with changes in trade practices, result 
in the need towiodlcally review, ahd possibly update. NOCTI exams. So that 
each reviewer may proceed with this task in a similar manner, the following 
criteria should be applied when making judgments regarding what should be- 
refnoved, added, or modified. 

REMOVED 

Items or jobs which are no longer found in general practice (found in 
less than 25 percent of the trade). 

ADDED 

Items or jobs which .are gaining acceptance in at least 25 percent of the 
trade. 

MODIFJED 

Changes within existing items or jobs which correct inaccuracies, improve 
clarity, increase precision, or reflect current practice more appropriately. 

When an item or job meets one or more of these criteria, the following 
steps should be taken: 

1. Identify the item by test occupation, item number, and page number. 

2. State briefly the nature of the problem with the ,i|£em. 

. 3. Describe fully the recommended change. The new item shoMld be 
presented in the eXact form that it is to appear on thetest. 

4. State how the replacement item tends to improve the exam. For 
example: What knowledges, understandings, or skills are tested by 
the new item? ■ • 

5. A multiple-choice format, offering fou*" responses per item, is 
required for written test items. . 
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. 6. Where fobs ar^ being recommended for a performance test, the following 
details are also needed: 

A. Description of job to be accomplished. 

B. Criteria for determining: 

1. The processes used (at least four) 
11. The results achieved (at least four) 

C. A value for each criterion on a five-point scale (5-25) 
IT. List of materials required. ' ' 

E. List of tools and equipment needed. 

F. If a technical drawing Is needed* It should be Included. 

6. Any special instructions needed by the evaluator for set-up or 
test administration. 
7; When a job Is being replaced, the substJ»itute task Should come as 

close to the original task as possible. In terms of time required to' ^ 
accompl ish'the job, total point value, and costs for materials. The 
new task should test skills and understandings, of Importance to the 
^ trade, which can only be perfected with years of experience In the 
occupation. 

COMPETENCY VS. CURRENCY 

When contemplating a change. It- Is Important to keep In mind that certain 
practices, although current, may not be used widely In; the field. Also, some 
practices which have been used for a long time may still be widely applied In 
fhe field. There are no hard and fast rules to apply here, although when In 
^ doybt, apply the 25 percent cr1ter1(jj|^. If In doubt, make a recommendation. 

Your recommendation will then be subjected to additional reviews at a later date. 

INNOVATIVE APPROACHES ' - ^ 

There will be times when some innovative approach to question formulation 
may be In order. When this occurs, the 1tem(s) should be formulated and a 
justification given describing the benefits of 'the new approach. For example, 
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it may bfr possible to think of a job which could be substituted for one or more 
Jobs. Efficiency and effectiveness are criteria that apply In these cases. The 
new item also might be constructed to measure characteristics not covered on 
the present exam but which are deemed essential. 

It will be easier to Implement changes wh.ich are formatted similarly to the 
original test items, but this should not precludejal ternate approaches if their 
benefits appear to be great enough to justify the innovation. 

/ 

When the. reviews have been comple-ted, all materials, Including test booklets 
and recommendations, should be returned to: 

Test Development Section 
NOCTI 

^- 45 Col vin Avenue 

Albany. NY 122Q6 

I 

\ 
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ATTACHMENT II 

DUTIES AND RESPONSIBILITIES 
OCCUPATIONAL COMPETENCY EVALUATION COORDINATOR 



DUTIES AND RESPONSIBILITIES 
OCCUPATIONAL COMPETENCY EVALUATION COORDINATOR 



♦ 

File signature with the Director of Teacher Certification, Bureau of 
Certification, (par. Ill B (3), Policies and General Procedures 
Relating to the Certification of Vocational Teachers in Pennsylvania), 

R^ulrement for the following is contained iu the Pennsylvania Policy 
Manual for Administration of the Occupational Competency Assessment 
Program for Vocational Instructional Certification Candidates and 
Vocational Intern Candidates. 

Prepare and disseminate, as necessary, an OCE brochure of instructions 
for candidates. 

Accept applications from prospective candidates. 
Review applicant credentials. 

Counsel all applicant^ ' i 

a . assessment procedure 

b. formal application. ' 

c. appropriate fee 

d. scheduled testing 

e . documents 

(1) education - ^ 

(2) work exjierience ^ * 

(3) occu{ia,tlonal licenses 

Maintain secure inventory of state examinations and related documents. 
Confidentiality and prevention of compromise is par amour\t/ 

Order required examinations and documents from, the State OCE Coordinator 
Administer the written examination. 

Arrange for cost f reO^ facilities and equipment for performance testing* 

Maintain current list of examiner's and status of in-service examiner 
training. 

F^*irtici4>ate in performance examl^tier training workshops. 

Engage examiners for testing. ' ^ 

Notify candidates of time, place and^other details of scheduled tasting 
by first c^laSvS mall. 

Review examiner's reports of perfonnahce testing. 

Determine candidate pass/fall by evaluating written arid performance 
tedt scores in relation to statistical data an(^ examiner's recommenda- 
tion. 

• ^ 70 ^'^ 
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16. Notify failed candidate, In writing , of failure, to Include diagnostic 
report of strengths and weaknesses and recommended corrective action 
If failed candidate should desire retestlng. 

17. Counsel* failed candidate through personal counseling period. 

18. Provide diagnostic report of strengths and weaknesses to passing 
candidate. If requested. 

19. Arrange payment of each examiner, reimburse supplies, etc. 

20. Maintain continuous detailed records of costs to be submitted annually 
tQ the Bureau of Vocational Edueatlon for cost anal y«*3 study of 

- statewide program. 
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Submit annual report to Bureau of Vocational Education on applications, 
persons^.tested, type assessment, pass/fall, etc. 

Represent university at state and national occupational competency 
meetings, seminars, etc. 

23. In absence of approved testing Instruments, organize and chair oral 
eKaralnatlon committee. Must include one occupational incumbent of- 
journeyman level. 

2A. Accept appeals of failed candidates and resolve- action pertaining 
thereto. » • 

25. Order NOCTI tests from NOCTI and arrange financial transactions, 

26. Arrange and host OCE Consortium meeting at least once annually and 
attend quarterly meetings of Consortium. 7^ 

27. Participate in determining and recommending test development. 

28. Participate in recommending OCE policy to state director of Vocational 
Education. 

29. Forward completed candidate optical scan sheets to the State OCE 
Coordinator for development of annual report of normative data 
subiiequently used for state norms. 



Forwjgrd completed summary sheet of candidate performance seeing to 
the iStat? OCE Coordinator for development of annual report of 
normative data subsequently used for state norms. 
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ATTACHMENT III 
PERSONNEL RESPONSfBILITY 
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OCE ENQUIRY - APPLICATION . ' 

A 

' .1 

1. Initial contact - Secretary ' :10 

2. Initial appointment with coordinator - Coordinator l-OO 

3. Mail application, forms - Secretary :15 

4. deceive application, prepare file - Secretary :10 

5. Receive, deposit and account for fees - Secretary '^^'^ ^'^'^''-v^^ 




Sir 
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6. Compile complete data, send follow-up correspondence - 

Secretary ' , ^ ' " 1:00 

, 7. Review file for completeness - Secretary :20 

8. Schedule and proctor pre-test and candidate experience 

claecklist comp-letlon - Secretary - - 1:00- 

9. Score pre-test and prepare full file for coordinator/ 

candidate interview - Secretary •• :30 

10. Candidate interview - Coordinator . . 2'. 00 

11. Order NOCTI examinations - Secretary • • ' ;10. 
..12. Order Pennsylvania examinations ^ Secretary ^ .:10 

13, Schedule testing Secre^ar^^^^oordinator / ' :1,5 

14. Prepare written testing packagT\ Secretary ■}» '^^ 
15/ ■ Prepare performance test ing^ckade and mail - 'Secretary -. ^ . :30 

16. Nptify candidate of examinatioX^edule, provide • 

instructions, maps, etc. - S^eretary ' :30 

17. . Proctor wriften test - Coordinator ' 4iOQ 
,18. Coordinate performance test - Coordinator » 10:00 

19. Score written exam. - Secretary - , . ' *:15 

20. Diagnostic review - Coordinator - ■ ^ ; -IB 

21. Receive performance results - Secretary , ^ . " ^ ;1P 

22. " Review performance resul ts - Coordinator " , - - :30 

23. ■ .Determine pass-fail ^nd written report - Coordinator.^ s - « 1:00 

24. Notify. candidate - Secret^r^y " ,^ , - :15 
• 25. Prepare certificate - Secr^|tary , .Qg 

26. .Prepare honorarium requests' for examipers"^ Secrlfetary :10 

27. Prepare payment for consumable supplies - . :15 

28. Failed CandidatjT- - ✓ 

■ / ■. ' , / 

Prepare file - 5«fcretary ^1:00 

' . . Interview candidate - Coordinator,, * 1:00 

Follow-up interview(s) ^ Coordinator . . • :3b 



